Learning models with uniform performance via distributionally robust optimization

A common goal in statistics and machine learning is to learn models that can perform well against distributional shifts, such as latent heterogeneous subpopulations, unknown covariate shifts or unmodeled temporal effects. We develop analyze a distributionally robust stochastic optimization (DRO) framework learns model providing good performance perturbations the data-generating distribution. give convex formulation for problem, several convergence guarantees. prove finite-sample minimax upper lower bounds, showing robustness sometimes comes at cost rates. limit theorems learned parameters, where we fully specify limiting distribution so confidence intervals be computed. On real tasks including generalizing fine-grained recognition tail performance, approach often exhibits improved performance.